1. Field of the Invention
This invention relates to an image processing apparatus and method. More particularly, the invention relates to an image processing method and apparatus for extracting image segments having different characteristics from an input color image and judging each of the characteristics of these image segments.
2. Description of the Related Art
Apparatuses for inputting and outputting color images have begun to find wider use in the field of office automation. This has been accompanied by greater opportunities for handling color images obtained by using an input device such as a scanner to read an original of a color image the greater part of which is occupied by a black-and-white portion or a portion wherein the number of colors is limited. Such a color image shall be referred to as a xe2x80x9cdocument imagexe2x80x9d. A feature of a document image is that, unlike other images, a document image includes a mixture of image segments (such as character image segments and photographic image segments) having different properties. There are cases in which an image having good picture quality is not obtained even if use is made of processing techniques utilized in other fields, such as a technique for zooming or a technique for reducing the quantity of data. Further, there is increasing demand for re-utilizing digital data obtained by structuring the color image of an original. More specifically, this entails inputting, into a computer, characters and photographs from the original in the form of character codes and image data, respectively, whereby the inputted information can be re-utilized in, say, desktop publishing (DTP).
Next, a technique for subjecting an image to zoom processing, a technique for reducing the quantity of data and a technique for image segmentation (inclusive of a technique for structuring a document image) will be described.
[Zoom Processing]
Methods of zooming an image (inclusive of converting pixel density) include selective processing conversion (SPC), in which each pixel of an original image is periodically eliminated in dependence upon the zoom ratio (this is image reduction) or in which each pixel is repeated in dependence upon the zoom ratio (this is image enlargement) (see Matsumoto, Kobayashi: xe2x80x9cStudy on the Effect of the Resolution Conversion on Picture Qualityxe2x80x9d, The Journal of the Institute of Image Electronics Engineers of Japan, Vol. 12, No. 5, pp. 354-362, 1983), and a method of obtaining the value of a pixel of interest by interpolation from neighboring pixels following a coordinate transformation (an affine transformation, etc.).
Further, a method proposed by the present applicant is available as a method of zooming a binary image. This method relates to an apparatus for extracting, smoothing and zooming outline information of a binary image and involves extracting outline vectors from a binary image, creating outline vectors that have been smoothly zoomed to a desired magnification (arbitrary) in the state of the extracted outline vector expression, and reproducing the binary image from the smoothing zoomed outline vectors, thereby obtaining a high-quality digital image zoomed to the desired magnification (arbitrary).
[Reduction of Data Quantity]
An example of an available method of reducing the quantity of image data when the image is stored or communicated is a coding method using a discrete cosine transform (DCT) according to the Joint Photographic Experts Group (JPEG). In another proposed method, a color image is converted to a uniform color space, after which the image is split up using color difference. The shape of a segment is subjected to chain coding, hue is coarsely quantized segment by segment using stored colors, saturation is subjected to a polynomial approximation and lightness is subjected to adaptive DCT coding, whereby the image data is coded. (See The Transactions of the Institute of Electronics, Information and Communication Engineers, B-I, Vol. J75-B-I, No. 6, pp. 422-430, 1992, June, 1992.)
Further, an example of a method of reducing the quantity of data in a binary image is MR/MMR coding or coding in accordance with the Joint Bi-Level Image Group (JBIG).
[Image Segmentation (Inclusive of Structuring of a Document Image)]
An example of a method of segmenting a color image is a recursive threshold-value method (see xe2x80x9cStructuring of Color Image Information by Area Partition Processingxe2x80x9d, Journal of Information Processing Society of Japan, Vol. 19, No. 12, pp. 1130-1136, Dec. 1987). According to this method, processing for deciding a threshold value for area partitioning from the value of a histogram in RGB, HSI, YIQ color features and then extracting an area is recursively repeated with regard to each extracted area. Another reported method is the limited-color expression method. This utilizes a color quantizing technique in which a color distribution is partitioned repeatedly until attainment of a number of colors in which a color error cannot be visually recognized by the human eye (see The Institute of Electronics, Information and Communication Engineers, 1990, Spring, All-Japan Convention, D146 7-168).
Examples of a method of segmenting image segments applied to a document image is one in which areas are judged pixel by pixel using the information possessed by neighboring pixels (see Tetsuya, Ochi: xe2x80x9cBilevel Rendition Method for Documents Including Gray-Scale and Bilevel Imagexe2x80x9d, The Transactions of the Institute of Electronic and Communication Engineers, Vol. J67-B, No. 7, pp. 781-788, 1984), and one in which an image segment is segmented by utilizing the background of a document image based upon the property that an image segment is partitioned by the background, after which the area is subjected to judgment (see Ito, et al: xe2x80x9cParallel Field Segmentation of Free-Format Documentsxe2x80x9d, 1979 Information Processing Society of Japan, 20th All-Japan Convention, pp. 453-454). Some of these papers also propose applying the results obtained by image segmentation to the structuring of document images. (For example, see Yamata, et. al: xe2x80x9cMultimedia Document Structuring Processing Systemxe2x80x9d, The Journal of the Institute of Image Electronics Engineers of Japan, Vol. 19, No. 5, pp. 286-295, 1990).
However, the problems described below arise in the techniques described above.
[Problems with Zoom Processing]
The SPC zoom method is capable of forming a zoomed image in simple fashion but problems such as jaggies arise in terms of image quality.
Further, a zoom method using coordinate transformation is effective for natural images, such as images of scenery. However, when this method is applied to document images, the edges of character or line drawing portions such as characters and graphs become blurred owing to interpolation processing.
Further, the method of zooming a binary image proposed by the present applicant is very effective as a method of extracting, smoothing and zooming the outline information of a binary image. However, this method cannot be applied directly to zooming of a color image or grayscale image.
[Problem with Data-quantity Reduction]
The method of reducing the quantity of data by coding using a DCT is effective for a natural image. However, when a document image is compressed and reproduced, image quality markedly deteriorates at character or line-drawing portions such as characters and graphs.
The method of segmentation and coding a segment is also for application to natural images and lightness is therefore subjected to adaptive DCT coding. As a result, the lightness information comes to occupy a major part of the total quantity of information. Accordingly, this method is useful with regard to continuous tone portions of a color image. However, coding efficiency is poor with regard to documents, in which black-and-white portions or limited-color portions occupy the major part of the document and the importance of lightness information is low.
The coding method based upon MR/MMR or JBIG is effective when reducing the amount of data in a binary image. However, this method cannot be directly applied when reducing the amount of data in a color image or grayscale image.
[Problem with Image Segmentation]
The method of image segmentation using the recursive threshold-value method or limited-color expression method can be applied to portion segmenting of a natural image. However, the method cannot be directly applied to a document image having image segments whose properties differ from those of a natural image (because the definition of a character segment is different from that of a natural image).
Further, the image segmentation method (inclusive of document structuring) for application to a character image is for the purpose of processing a binary image or a grayscale image. It is difficult to apply this method to a color image in which there is much overlapping of image segments.
Thus, the proposed zoom method, data reducing method and image segmentation method are often suitable for natural images but satisfactory results cannot be expected when these methods are applied to a document image. Further, the proposed methods for application to document images are suitable for binary images and grayscale images but cannot be directly applied to color images.
An object of the present invention is to solve the aforementioned problems and provide an image processing apparatus and method in which excellent processing is applied to a color image in which image segments having different characteristics are mixed.
According to a preferred embodiment of the present invention, the foregoing object is attained by providing an image processing method comprising an input step of inputting a color image, an extraction step of extracting image segments, which have characteristics different from a background image segment of the inputted color image, from the color image, and a discrimination step of discriminating the characteristics of each image segment extracted at the extraction step.
The invention further discloses an image processing method for extracting each image segment from a color image in which image segments having different components are mixed, comprising a creation step of creating a reduced image from an input color image, a first extraction step of extracting an image segment from the reduced image, and a second extraction step of extracting an image segment from the input color image using data of the image segment extracted at the first extraction step.
The invention further discloses an image processing apparatus comprising input means for inputting a color image, extraction means for extracting image segments, which have characteristics different from a background image segment of the input color image, from the color image, and discrimination means for discriminating the characteristics of each image segment extracted by the extraction means.
The invention further discloses an image processing apparatus for extracting each image segment from a color image in which image segments having different components are mixed, comprising creation means for creating a reduced image from an input color image, first extraction means for extracting an image segment from the reduced image, and second extraction means for extracting an image segment from the input color image using data of the image segment extracted by the first extraction means.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.